Towards a systematic analysis of cluster computing log data: the case of IBM BlueGene/Q

The complexity and cost of managing large computing infrastructures is on the rise. Automating management actions to minimize human intervention is an attempt at containing these costs. This in turn requires building models and making predictions through analysis of log data. In current studies, this analysis is often limited to restricted datasets, and as such, is not applicable at a system level. Building models that are accurate enough to be useful in real systems requires analysis of log data from disparate sources. In this paper we provide a characterization study with four datasets reporting on power consumption, temperature, workload and hardware/software events for an IBM BlueGene/Q cluster. We show that the system handles a very rich parallel workload, with low correlation among its components in terms of temperature and power, but higher correlation in terms of events. Power and temperature correlate strongly, while events display negative correlations with load and power. Power and workload show moderate correlations, and only at the level of components. The aim of the study is a systematic, integrated characterization of the computing infrastructure and discovery of correlation sources and levels to be used in future predictive modeling efforts. Keywords-BlueGene/Q; log data analysis; correlation; data integration; characterization